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Evaluation  of  a  Videodisc  Delivery  System 
For  Teaching  Students  to  Troubleshoot 
The  AN/VRC-12  Medium  Powered-Radio  Series 

Executive  Summary 

Random  assignment  was  made  of  two  hundred  thirty-five  trainees  to  two 
Instructional  groups.  Regression  analysis  showed  that  those  (n«72}  using  a 
videodisc  delivery  system  to  learn  Task  17  (lessons  1-7  of  No^le  8)  of  the 
Tactical  Communications  System  Operator/Mechanic  Course  reached  the  mastery 
criterion  with  a  significantly  lower  mean  progression  Index  than  that  of 
trainees  (n>163)  learning  the  same  materials  with  a  slide-tape  projector 
system.  Videodisc  trainees  and  Instructors  Indicated  (on  questionnaires) 
a  high  degree  of  acceptance  of  the  videodisc  system,  preferring  It  overwhelm¬ 
ingly  to  the  slide-tape  system.  On  Task  18  of  the  course,  all  trainees  used 
the  slide-tape  system,  and  the  videodisc  trainees  had  a  significantly  greater 
mean  completion  time  than  that  of  the  slide- tape  group.  It  can  be  concluded 
that  the  videodisc  system  was  superior  to  the  slide-tape  system  In  this 
situation,  although  the  complexity  of  the  study  makes  It  Impossible  to  explain 
the  nature  of  the  superiority  of  the  videodisc  system  or  to  generalize  It  to 
other  settings  or  even  to  the  entire  course.  Reasons  for  the  greater  mean 
time  of  videodisc  trainees  when  they  resumed  study  with  the  slide-tape  system 
were  not  discovered. 

Because  no  scheduled  maintenance  was  carried  out  and  there  was  only 
one  equipment  failure  during  the  data  collection  period,  no  comparison  of 
the  two  systems  for  maintenance  costs/problems  was  made. 
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Evaluation  of  a  Videodisc  Delivery  System  For  Teaching 
Students  to  Troubleshoot  The  AN/VRC-12  Medium- Powered  Radio  Series 

Introduction 


Background 


It  is  widely  recognized  by  the  Army  training  community  that  as  tasks 
required  of  military  personnel  become  more  complex,  the  need  for  increased 
effectiveness/efficiency  in  training  programs  becomes  correspondingly  more 
important.  Educational  technology  offers  one  possible  avenue  to  such  in¬ 
creases,  and  studies  of  several  kinds  and  combinations  of  instructional 
equipment  are  being  undertaken  at  this  time. 

Videodisc  systems  appear  to  have  great  potential  for  improving 
training  programs;  however,  in  a  study  of  performance  of  Armor,  Artillery, 
and  Infantry  soldiers  on  pairs  of  Army  Training  Extension  Course  (TEC) 
lessons  (Holmgren,  Dyer,  Hilligoss  and  Heller,  1979),  this  potential  for 
superiority  over  another  method  of  presentation  (the  Beseler  Cue/See  viewer) 
was  not  demonstrated.  Review  of  the  study  indicated  that  this  failure  to 
show  improved  training  effectiveness/efficiency  with  videodisc  delivery  may 
have  resulted  from  three  unrelated  causes:  frequent  equipment  and  program 
malfunctions,  short  treatment  times  (only  one  hour),  and  small  sample  sizes. 

The  current  study  was  designed  to  evaluate  a  videodisc  training 
delivery  system  under  conditions  where  the  causes  for  failure  listed  above 
could  not  pertain.  In  addition  to  determining  the  training  effectiveness/ 
efficiency  of  the  system,  the  study  was  designed  also  to  evaluate  the  accept 
ability  of  the  videodisc  system  to  trainees  and  instructors  as  well  as  the 
reliability  of  the  specific  system  used. 

Purpose 


The  purpose  of  this  document  is  to  report  the  results  of  an  evalua¬ 
tion  study  of  a  videodisc  training  delivery  system  used  in  teaching  students 
to  troubleshoot  the  AN/VRC-12  medium- powered  frequency  modulated  (FM)  radio 
series.  The  videodisc  system  used  nonprogrammable  Magnavox  VH8000  consumer 
videodisc  players.  The  instructional  module  on  medium- powered  FM  radios  is 
the  eighth  in  a  series  of  fourteen  modules  that  make  up  the  entire  Tactical 
Communications  System  Operator/Mechanic  Course  (MOS)  31V10.  This  module 
contains  fifteen  lessons,  two  enabling  examinations,  and  one  criterion 
examination.  The  videodisc  delivery  system  was  used  with  only  the  first 
seven  lessons  in  the  module  (Task  17).  They  were  tested  with  one  of  the 
two  enabling  examinations. 

The  objectives  of  the  evaluation  were  to  determine  the  training 
effectiveness/efficiency  of  the  videodisc  system,  its  acceptability  to 
students  and  instructors,  and  the  reliability  of  its  equipment.  Specific 
evaluation  questions  were  asked  concerning  the  first  two,  training  effec¬ 
tiveness/efficiency  and  student  and  instructor  acceptance;  acceptability 
of  the  system  and  equipment  reliability  were  evaluated  by  use  of  question¬ 
naires  and  records. 


The  use  of  highly  specific  questions  in  evaluation  or  comparison  of 
the  instructional  effectiveness/efficiency  of  different  delivery  systems 
serves  to  make  the  purposes  of  such  evaluation/comparison  entirely  clear. 
Questions  formulated  for  this  study  follow: 

1.  Will  students  who  learn  to  troubleshoot  med i urn- powered  FM 
radios  from  the  videodisc  delivery  system  attain  a  statis¬ 
tically  significantly  lower  mean  progression  index  IPI)V 
than  that  attained  by  students  who  learn  from  the  current 
system? 

2.  Will  the  difference  in  mean  Pi's  of  the  two  groups  be  of 
practical  significance,  i.e.,  will  a  treatment  effect  size 
of  at  least  -.2  Pi's  be  attained? 

3.  Will  videodisc  students  have  a  statistically  significantly 
higher  mean  on  a  knowledge  retention  test  than  that  of  students 
using  the  current  system? 

If  the  hypothesis  of  no  difference  between  the  PI  means  of  the 
two  groups  were  rejected  at  or  beyond  an  alpha  level  of  .05,  these  ques¬ 
tions  would  be  affirmatively  answered.  A  treatment  effect  size  of 
approximately  .2  Pi's  or  greater  would  indicate  that  adoption  of  the  video¬ 
disc  system  for  these  lessons  would  be  worthwhile. 

4.  Will  there  be  statistically  significantly  fewer  students 
in  the  videodisc  system  who  are  required  to  repeat  the 
module  than  in  the  current  system? 

5.  When  asked  to  indicate  the  degree  of  acceptance  of  the  system 
used  for  the  first  seven  lessons  of  the  medium- powered  FM  radio 
series,  will  students  who  receive  training  with  the  videodisc 
system  express  statistically  significantly  greater  acceptance 
than  will  students  who  receive  training  with  the  current  system? 
(This  question  was  to  compare  acceptance  of  the  same  lessons 
with  different  systems  and  different  subjects.) 

6.  Will  students  who  experience  both  the  videodisc  and  the  current 
delivery  system  express  greater  satisfaction  with  the  videodisc 
system?  (This  question  was  to  compare  acceptance  of  different 
lessons  with  different  systems  and  the  same  subjects.) 

Questions  3,  4,  and  5  would  be  considered  affirmatively  answered 
if  hypotheses  of  no  differences  between  proportions  or  means  were  rejected 


The  PI  is  the  amount  of  active  time  taken  by  a  student  to 
complete  the  lessons  in  a  task  divided  by  the  assigned  time  allotted 
for  successful  completion  of  that  task.  Active  time  is  time  actually 
spent  in  learning  the  material. 
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at  or  beyond  alpha  levels  of  .05. 


7.  Will  the  instructors  who  supervise  students  undergoing  instruction 

in  either  videodisc  or  current  systems  express  greater  satisfaction 

and/or  acceptance  of  the  videodisc  system  than  the  current  system? 

This  question  was  answered  through  a  qualitative  analysis  of  instruc¬ 
tor  responses  to  a  questionnaire  and  to  interviews.  No  statistical  analysis 
was  made  of  their  responses  because  of  the  small  number  of  instructors 
involved. 

Evaluation  Design 

The  target  population  for  this  study  consisted  of  students  who  have 
recently  finished  basic  combat  training,  who  have  normally  ten  or  more 
months  of  active  duty  service  remaining  after  completion  of  the  course,  and 
who  have  electronic  aptitude  scores  of  at  least  90  on  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB).  Two  hundred  thirty-five  students  from 
this  population  were  used  in  the  evaluation  study;  72  of  them  were  randomly 
assigned  to  receive  training  via  the  videodisc  system  while  the  other  163 
used  the  current  system.  The  unequal  numbers  resulted  from  the  fact  that 
only  8  videodisc  training  positions  were  available  while  there  were  19 
current  system  training  positions.  Two  backup  positions  equipped  with 
videodiscs  were  available  to  be  used  in  case  of  equipment  failure  or  for 
either  videodisc  or  control  students  if  either  set  of  available  positions 
were  fully  occupied. 

It  was  originally  planned  to  obtain  100  students  for  the  videodisc 
treatment  and  approximately  224  students  for  the  comparison  (covariate) 
treatment.  These  numbers  were  calculated,  given  an  estimated  effect  size 
of  -.2  Pi's  (a  difference  in  mean  student  learning  time  of  about  five  hours), 
alpha  =  .05,  and  statistical  power  of  .80.2  Because  of  time  pressures,  the 
total  number  required  could  not  be  obtained. 

Training  conditions.  The  current  Tactical  Communication  System 
Operator/Mechanic  Course  is  a  self-paced  program  that  utilizes  an  Individual 
Learning  Center  (ILC),  a  set  of  radio  and  basic  electronics  laboratories, 
and  three  testing  rooms.  The  learning  center  is  equipped  with  a  number  of 
student  carrels,  each  containing  a  Caramate  slide-tape  projector;  the 
laboratory  that  is  concerned  with  the  eighth  module  contains  a  number  of 
AN/VRC-12  radios  whose  components  can  be  modified  to  exhibit  various 
defects;  and  the  related  testing  room  contains  radios  identical  to  those 
in  the  laboratory. 

Upon  entering  the  program,  a  student  is  issued  a  Student  Guide 
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that  acquaints  him  with  the  program  and  the  procedures  to  be  followed. 

Using  a  printed  Guidance  Package  for  the  appropriate  module,  the  student 
works  through  a  lesson  by  receiving  instruction  from  the  Caramate  and  by 
completing  a  series  of  written  and/or  laboratory  exercises.  Enabling 
examinations  are  administered  to  the  student  at  the  end  of  each  lesson  or 
set  of  lessons  by  an  instructor  in  the  ILC,  laboratory,  or  testing  room, 
and  a  decision  is  made  to  determine  whether  the  student  will  continue  with 
the  next  lesson  or  repeat  the  current  lesson  or  set.  A  criterion  examina¬ 
tion  is  administered  in  the  testing  room  at  the  end  of  each  module,  at 
which  time  a  decision  is  made  as  to  whether  each  student  will  proceed  to 
the  next  module  or  retake  all  or  parts  of  the  current  one.  Student  progress 
is  recorded  in  terms  of  a  progression  index  (PI),  which  is  the  amount  of 
time  taken  by  the  student  to  complete  a  task  divided  by  the  assigned  time 
allotted  for  that  task. 

The  videodisc  delivery  system  was  used  for  only  the  first  seven 
lessons  of  the  eighth  module,  and  an  enabling  examination  was  given  to 
each  student  at  time  of  completion.  The  videodisc  system  differed  from 
the  currently  used  one  in  four  ways.  First,  the  videodisc  equipment  was 
used  instead  of  the  Caramate  projector.  Second,  the  videodiscs  were  placed 
in  the  laboratory  and  each  of  them  was  paired  with  an  AN/VRC-12  radio. 

Third,  the  courseware,  including  the  Guidance  Package,  for  the  seven  lessons 
was  completely  rewritten  to  take  advantage  of  the  videodisc  capabilities  for 
fast  and  slow  motion,  freeze  frame,  and  random  access.  The  subject  matter 
of  the  lessons  did  not  change  nor  were  the  enabling  tests  or  criterion  exams 
changed.  Fourth,  an  introduction  to  the  videodisc  system  was  presented  to 
the  student  via  printed  material  and  the  videodisc  itself  before  the  student 
began  the  first  lesson.  All  other  aspects  of  this  portion  of  the  course 
were  the  same. 

Instrumentation.  The  major  criterion  or  dependent  variables  used 
in  the  study  were  the  previously  defined  progression  index  (PI)  and  scores 
on  the  enabling  and  criterion  examinations.  These  variables  were  related 
in  that  students  who  received  "no  go's"  on  examinations  had  to  retake  all 
or  parts  of  modules,  thereby  increasing  their  Pi's.  It  was  necessary, 
therefore,  to  be  certain  that  these  variables  contained  sufficient  relia¬ 
bility  and  validity  to  be  sensitive  to  differences  in  the  two  delivery 
systems. 

Content  validity  is  of  primary  concern  when  achievement  or  pro¬ 
ficiency  tests  are  under  consideration.  In  assessing  the  content  validity 
of  a  test,  the  following  question  is  asked: 

Do  the  observations  truly  sample  the  universe  of  tasks  the 
developer  intended  to  measure  or  the  universe  of  situations  in 
which  he  would  like  to  observe?^ 
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The  question  is  answered  by  assessing  the  adequacy  of  the  set  of  operations 
used  in  test  construction.  These  operations  include  defining  the  domain  or 
universe  of  interest,  specifying  the  procedures  for  sampling  the  domain  and 
for  stimulus  and  response  construction,  planning  the  materials  and  instruc¬ 
tions  given  by  examinees,  and  formulating  the  rules  by  which  the  responses 
are  evaluated. 

The  first  seven  lessons  in  the  eighth  module  are  concerned  with 
six  pieces  of  equipment  that  can  fail  in  one  or  more  ways  and  that  make 
up  an  AN/VRC-12  series  radio.  The  six  pieces  of  equipment  and  the  number 
of  their  possible  defects  are  as  follows: 

No.  of 
Possibi  e 


AN/VRC-12  Radio  Equipment  Defects 


CS  4720  Cable  2 

MT  1029  Mounting  platform  1 

RT  Receiver  transmitter  3 

CX  4722  Cable  3 

MX  Matching  unit  for  antenna  3 

Source  1 
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The  enabling  examination  consists  of  supplying  the  student  with  an  Equip¬ 
ment  Inspection  and  Maintenance  worksheet  (Form  DA  2404),  assigning  him 
to  a  radio  with  known  defects,  requiring  him  to  discover  its  deficiencies, 
and  requiring  him  to  indicate  corrective  actions.  The  student  must  correctly 
identify  deficiencies  and  prescribe  proper  corrective  action  for  at  least  two 
of  three  problems.  One  problem  must  involve  the  matching  unit  (RT,  CX  4722, 
MX),  one  must  involve  the  power  input  (source,  CX  4720,  MT  1029,  RT),  and 
the  third  is  chosen  by  the  examiner.  It  is  highly  likely  that  if  a  student 
can  solve  two  problems  he  can  deal  with  all  the  problems  in  the  domain  since 
a  common  procedure  is  involved  in  all  of  them.  Well  defined,  objective  rules 
for  evaluating  the  student's  performance  exist.  Thus,  while  no  formal 
studies  of  inter-examiner  reliability  were  made,  it  is  believed  that  agree¬ 
ment  between  examiners  concerning  scoring  is  sufficiently  high  for  the 
purposes  of  the  evaluation  study. 

The  discussion  above  Indicates  that  the  enabling  exam  for  the  first 
seven  lessons  on  the  AN/VRC-12  has  high  content  validity;  however,  there 
could  be  some  question  about  the  stability  of  the  PI.  That  is,  the  active 
study  time  for  a  student  could  be  influenced  by  many  temporary  factors 
(e.g.  changes  in  motivational  and  emotional  states,  physical  well-being, 
etc.)  to  the  extent  that  fluctuations  in  it  could  make  it  insensitive  to 
differences  in  the  two  delivery  systems.  As  a  check  on  this  possibility,  PI 
data  were  gathered  on  the  first  7  and  the  next  5  lessons  of  the  AN/VRC-12 
module  for  29  graduates  of  the  course.  The  Pearson  product  moment  correla¬ 
tion  between  the  two  PI  variables  was  .73.  This  moderately  high  degree  of 
agreement  between  Pi's  taken  at  two  different  times  suggests  that  the 


stability  of  the  PI  is  probably  sufficiently  great  to  allow  it  to  detect 
differences  in  the  two  delivery  systems. 

Concerns  about  content  validity  of  the  enabling  examination  which 
follows  the  second  set  of  lessons  in  the  module,  and  the  criterion  examina¬ 
tion  which  is  administered  to  students  when  they  have  completed  the  module 
are  the  same  as  those  for  the  first  enabling  exam.  The  rationale  given  above 
would  thus  apply  equally  to  these  tests. 


Since  there  could  be  some  question  about  the  ability  of  the  pass/fail 
score  of  the  enabling  exam  and  the  PI  to  detect  differences  in  the  two 
delivery  systems  a  knowledge  retention  test  was  developed  and  used  as  a 
criterion  variable.  This  test  contained  40  items  selected  from  exercises 
in  the  guidance  packages.  The  table  given  below  shows  for  the  first  seven 
lessons  in  the  current  AN/VRC-12  radio  guidance  package  the  number  and 
proportion  of  potential  test  questions  each  contains  and  the  number  of 
questions  which  were  chosen  from  each  lesson  through  proportional  random 
sampl ing  procedures . 


Lesson 


8-2-1 

8-2-1 .1 

8-2-2 

8-3 

8-4-1 

8-4-2 

Total 

Number  of 
Questions 

16 

9 

6 

n 

45 

22 

22 

139 

Proportion 

.12 

.06 

.04 

H 

.32 

.16 

.16 

1 .00 

Number  of 
Test  Items 

5 

2 

2 

B 

13 

6 

6 

40 

Figure  1.  Distribution  of  Knowledge  Retention  Test  Items 


This  test  appears  to  meet  the  operational  criteria  for  high  content  validity. 
It  was  designed  after  the  videodisc  courseware  had  been  developed  and  was 
constructed  in  such  a  way  as  to  have  high  content  validity  for  both  sets  of 
courseware.  A  copy  of  it  is  given  in  Appendix  A. 

The  three  instruments  that  were  used  in  studying  the  human  acceptance 
of  the  videodisc  system  are  also  presented  in  Appendix  A.  The  first  two  are 
questionnaires  that  were  administered  to  students,  one  to  those  who  learned 
from  the  Caramate  del ivery  system  and  the  other  to  those  who  used  the  video¬ 
disc  system.  The  third  is  a  questionnaire  for  instructors. 

Items  1  -  17  on  the  student  questionnaires  were  identical  except 
for  location  designation,  and  scores  on  them  were  used  to  compare  acceptance 
of  the  two  systems.  Individual  item  responses  were  used  to  help  explain 
differences  in  effectiveness  that  might  occur.  Items  18  -  21  on  the  Caramate 
student  questionnaires  to  discover  whether  there  were  factors  in  the  physical 
environment  in  the  ILC  that  could  possibly  have  affected  learning  efficiency. 
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Each  of  items  18  -  27  on  the  questionnaire  for  videodisc  students  was 
examined  separately  in  studying  the  relative  acceptability  of  the  two  systems. 
Both  of  these  instruments  were  administered  to  students  who  passed  the 
enabling  exam  on  the  first  seven  lessons  of  the  AN/VRC-12  radio.  Administra¬ 
tion  of  them  was  carried  out  by  instructors  in  the  ILC  before  students  begin 
the  next  set  of  lessons.  The  questionnaire  for  1ab  instructors  was  completed 
at  the  end  of  the  10-week  evaluation  session.  In  addition,  these  instructors 
were  asked  to  keep  records  of  instances  of  unusual  effectiveness  or  difficulty 
displayed  by  either  system  during  the  10-week  period. 

Both  student  questionnaires  are  essentially  expanded  versions  of  one 
that  was  used  In  formative  evaluation  of  this  course.  A11  three  Instruments 
provide  space  for  informal  comments  for  each  item  and  for  overall  reactions. 

In  addition  to  the  treatment  classification,  independent  variables 
were  education  (highest  grade  completed)  from  the  Report  of  Student  Inter¬ 
views  (FS  Form  41 2),sel f-estimate  of  reading  ability  from  the  Student 
Personal  History  (FS  Form  58),  and  general  technical  (GT)  and  electronic 
aptitude  (EL)  scores  on  the  ASVAB.  These  two  forms  are  completed  by  all 
students  when  they  enter  the  course.  ASVAB  scores  were  obtained  from  the 
students'  personnel  records. 

Design.  A  true  experimental  design  was  employed  in  the  evaluation 
study  since  students  were  randomly  assigned  to  treatments.  It  Is  shown  In 
diagrammatic  form  below: 


Week  1 
R  X..  I 


R  X_  0.. 


Week  2 


R 

R 


0 

0 


1 

2 


Week  10 

R  X,  0, 

R  X^  Oj 


Figure  2.  Experimental  Design 


In  the  diagram,  R  Indicates  random  assignment  to  either  the  videodisc 
system  (Xy)  or  the  Caramate  system  (X^).  and  &i  and  Oo  are  the  criterion 
var tables  taken  for  each  group.  The  design  was  repeated  for  10  weeks  until 
72  students  had  received  training  on  the  videodisc  system  and  163  had  re¬ 
ceived  training  with  the  Caramate  system. 

Data  Collection  and  Verification 

Current  kinds  of  data  and  methods  of  collection  with  only  a  few 
changes  were  adequate  for  evaluation  purposes.  At  the  beginning  of  each 
module,  each  student  was  issued  a  student  guidance  card  (FS  Form  1051)  on 
which  the  Instructor  recorded  pass/fail  data  and  beginning  and  ending  times 
for  each  lesson.  These  forms  were  collected  for  the  entire  ten-week  course 
and  were  used  to  construct  data  files.  Three  computer -gen era ted  reports  were 
produced  from  these  files.  The  first,  and  most  important  for  the  evaluation, 
reports  for  each  student  the  time  he  spent  on  each  of  31  tasks  which  are 
made  up  of  blocks  of  lessons  followed  by  enabling  or  criterion  examinations. 
This  report  also  includes  the  average  time  spent  on  each  task  by  all  the 
students  in  the  class  and  the  assigned  time  for  each  task.  The  second  report 
(Class  Status  Roster)  gives  for  each  student  for  the  total  course,  the  PI, 
status  (Graduated,  Failed,  still  in  progress),  number  of  tasks  completed  and 
the  date  of  the  last  task  completed.  It  then  gives  the  assigned  time  fbr 
the  number  of  tasks  completed,  the  average  time  of  all  students  in  the  class, 
the  active  and  administrative  times  for  each  student,  and  the  estimated 
number  of  days  to  graduation.  The  third  report  (Current  Status  Report)  is 
a  summary  of  the  performance  of  a  series  of  classes.  It  lists  for  each 
class  the  class  number,  the  starting  data,  the  number  of  starting  and  remain¬ 
ing  students,  the  number  and  percentage  of  students  relieved  because  of 
administrative  or  academic  reasons,  the  number  and  percentage  of  turnbacks, 
the  number  and  percentage  of  graduates,  the  average  number  of  days  and  hours 
to  graduation,  and  the  total  number  and  percentage  of  reliefs. 

In  addition  to  the  basic  data  provided  by  the  guidance  cards, 
examination  results  were  kept  by  the  AN/VRC-12  examiner.  These  consisted 
of  the  equipment  inspection  and  maintenance  worksheets  (Form  DA  2404)  and 
pass/fail  logs.  These  data  together  with  the  report  of  student  interviews 
(FS  Form  412)  and  the  student  personal  history  (FS  Form  58)  were  retained 
for  evaluation  of  students.  Copies  of  all  forms  and  the  first  computer¬ 
generated  report  are  shown  in  Appendix  B.  Forms  for  gathering  reliability, 
maintainability  and  cost  data  for  the  videodisc  equipment  are  shown  in 
Appendix  C. 

Data  that  were  gathered  for  evaluation  purposes  only  were  scores 
on  the  knowledge  retention  test  taken  by  students  upon  passing  the 
enabling  exam  on  the  first  seven  lessons  of  the  AN/VRC-12  and  the  student 
and  instructor  questionnaires. 

Two  data  files  were  prepared.  The  first  contained  for  each  student 
an  ID  number  and/or  name;  group  membership  (1  if  a  member  of  the  videodisc 
group,  0  otherwise);  highest  grade  of  school  completed;  student  estimate 
of  reading  ability  (codes  1  through  4);  ASVAB  GT  and  EL  scores;  active 
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student  times  for  the  first  seven  lessons  of  the  AN/VRC-12  module  (Task  17), 
the  second  five  lessons  of  the  module  (Task  18),  and  the  last  three  lessons 
of  the  module  (Task  19);  pass/fail  scores  on  the  enabling  and  criterion  exams 
of  the  AN/VRC-12  radio;  total  score  (number  correct)  on  the  knowledge  reten¬ 
tion;  and  total  scores  from  items  1  -  17  on  the  student  questionnaire.  This 
data  file  was  entered  directly  into  the  computer  through  a  terminal  from  the 
data  collection  form.  The  second  data  file  was  constructed  from  the  student 
questionnaires.  The  file  was  made  with  individual  item  responses  in 
columns  1  -  32  for  the  Caramate  students  and  columns  1  -  43  for  the  video¬ 
disc  students. 

The  following  verification  procedures  were  used  after  any  data 
collection  step  involving  computation,  recording,  or  file  preparation  by 
hand:  (1)  Select  a  random  sample  of  approximately  10%  of  the  data  set. 

(2)  Have  a  person  who  did  not  do  the  original  work  recompute  or  re-record 
the  data  from  the  sample.  (3)  If  more  than  2%  errors  are  found  (disregard¬ 
ing  round-off  and  other  minor  errors)  check  each  data  record  and  correct 
errors.  (4)  Check  each  student  time  for  Tasks  17,  18,  and  19  that  deviate 
markedly  from  the  assigned  time.  Deviant  scores  were  examined  for  record¬ 
ing  or  computation  errors  and  corrected  if  necessary. 

4 

Data  analysis.  Regression  analysis  was  the  major  method  used  In 
analyzing  the  data.  Its  purpose  was  to  determine  the  statistical  signifi¬ 
cance  of  the  difference  between  the  mean  Pi's  of  the  two  groups  while 
holding  the  effect  of  amount  of  education  constant.  The  basic  model 
employed  is  shown  below: 

Y  .  a  >  b^X^  +  b2X2  +  bjXj 

A 

where  Y  is  the  predicted  value  of  a  criterion  variable,  X^  is  the  dummy 
coded  group  membership  variable  (videodisc  >  1,  Caramate  =  D,  X2  Is  the 
education  score  from  the  Report  of  Student  Interviews, 5  X3  is  the  Inter¬ 
action  of  the  two  independent  variables  (X1X2),  the  b's  are  regression 
coefficients,  and  a  is  a  constant.  The  first  analysis  used  the  PI  for 
Task  17  as  the  dependent  variable  and  the  data  were  fitted  to  the  model. 

The  statistical  significance  of  b3  was  checked  using  an  F-test;  since  It 
was  not  different  from  zero,  that  term  was  dropped.  The  analysis  was 
redone  using  as  the  basic  model  ?  =  a  +  b]  X-)  +  b2X2,  and  the  significance 
of  b-j  was  determined.  Since  b]  was  negative  and  nonzero,  the  videodisc 
system  was  declared  to  be  more  effective  than  the  current  system.  Df 
secondary  interest  was  the  statistical  significance  of  b2;  it  was  nonzero. 
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Kerlinger,  F.  N.  and  Pedhazur,  E.  J.,  Multiple  Regression  in 
Behavioral  Research.  (New  York:  Holt  Rinehart  and  Winston,  1973.) 
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In  the  original  evaluation  plan  ASVAB-GT  was  specified  as  the 
covariate  in  the  regression  analysis.  Because  GT  scores  for  forty-nine 
subjects  could  not  be  obtained,  it  was  decided  to  substitute  years  of 
education  as  the  covariate. 


and  the  conclusion  was  that  the  magnitude  of  PI  Is  related  to  the  educa¬ 
tional  attainments  of  the  subjects.  Since  both  b]  and  b2  were  significant, 
the  basic  equation  was  decomposed  to  yield  an  equation  for  each  group,  for 
the  videodisc  group  it  was  Y  -  (a+b] )  -i-  b2X2,  and  for  the  Caramate  group  It 
was  ?  =  a  +  b2X2.  Thus,  b-j  can  be  interpreted  as  the  difference  betweefn  the 
means  of  the  two  groups  adjusted  for  initial  group  differences  in  educational 
1 evel . 

Patterns  of  correlations  among  the  independent  variables  were 
examined.  It  did  not  appear  that  by  using  one  or  more  of  the  other  inde¬ 
pendent  variables,  significant  increments  in  the  amount  of  explained 
variance  of  the  dependent  variable  would  be  attained. 

The  second  and  third  regression  analyses  used  as  dependent  variables 
total  scores  on  the  knowledge  retention  test  and  total  scores  on  the  first 
seventeen  items  on  both  student  questionnaires.  Chi-square  tests^  were 
used  to  test  differences  in  pass/fail  frequencies  on  the  enabling  exam  and 
to  determine  differences  in  frequencies  of  responses  to  items  that  compare 
the  videodisc  and  Caramate  systems  where  appropriate. 

Additional  analyses  used  the  basic  regression  model  to  test  for 
system  transfer  effects  using  as  the' dependent  variable  the  PI  of  the 
second  set  of  lessons  on  the  AN/VRC-12  radio  (Task  18). 


Discussion 


Conduct  of  the  Study 

Project  start-up  procedures  included  designation  of  a  person 
responsible  for  seeing  that  necessary  data  were  collected  and  stored 
for  later  shipment  to  the  evaluator,  accumulation  of  Class  Status  Rosters 
and  Current  Status  Reports  for  later  use  as  comparison  data,  briefing  of 
participating  personnel  on  the  videodisc  system  and  the  evaluation 
project,  and  delivery  to  participating  personnel  of  necessary  forms  and 
instructions  for  their  use.  FS  Forms  412  and  FS  Forms  58  were  collected 
for  all  students  in  participating  classes. 

Assignment  to  the  videodisc  or  the  Caramate  group  was  made  for 
participating  students  at  the  time  they  passed  the  criterion  examination 
for  module  seven.  A  previously  determined  random  assignment  schedule  was 
used  for  this  purpose.  Group  assignments  were  recorded  on  FS  Forms  1051 
and  also  on  Class  Rosters  for  the  Evaluation  Study. 

When  each  participating  student  passed  the  enabling  examination 
for  Task  17,  both  the  knowledge  retention  test  and  the  appropriate  ques¬ 
tionnaire  were  administered.  Test  scores  and  completed  questionnaires 


®See  any  basic  statistics  text;  for  example,  A.  L.  Edwards, 
Statistical  Analysis.  4th  ed.  (New  York:  Holt  Rinehart  and  Winston,  1974). 


Mere  stored.  In  addition,  ASVAB  GT  and  EL  scores  were  obtained  and 
stored  on  appropriate  forms  for  all  students  who  entered  the  program, 
regardless  of  whether  they  completed  it. 

For  each  participating  class,  the  following  data  sources  were 
obtained  and  stored: 

(1)  DA  Forms  2404  for  Tasks  17,  18,  and  19 

(2)  Pass/Fail  rosters  for  Tasks  17,  18,  and  19 

(3)  Originals  of  FS  Forms  1051  for  each  student 

(4)  Verified  computer  printouts  for  each  student 

(5)  Class  status  rosters 

Following  completion  of  the  course  by  the  last  participating 
student,  a  printout  of  the  current  status  report  was  obtained  and  stored. 
Checks  were  made  to  ensure  that  all  participating  classes  were  listed  and 
that  records  were  complete. 

Lab  Instructor  questionnaires  were  administered  and  collected  as 
soon  as  the  last  participating  student  passed  the  enabling  examination  for 
Task  17.  Reliability  and  Maintenance  and  Cost  Data  Forms  from  maintenance 
personnel  were  collected  at  the  same  time. 

Findings 


The  results  of  the  study  are  presented  in  this  section.  Descriptive 
statistics  are  given  first  and  are  followed  by  the  results  of  analyses 
specifically  designed  to  answer  the  evaluation  questions  stated  in  the 
Purpose  section.  Selected  computer  output  is  given  in  Appendix  D. 

Table  1  gives  the  means  and  standard  deviations  of  independent  and 
dependent  variables  for  both  treatment  and  control  groups.  It  can  be  seen 
that  the  randomization  procedure  produced  groups  that  are  reasonably  equiv¬ 
alent  on  the  independent  variabl es--education,  self-estimate  of  reading 
ability,  ASVAB-6T,  and  ASVAB-EL.  Although  the  Caramate  group  was  slightly 
superior  on  each  of  the  variables,  none  of  the  differences  was  statistically 
significant. 

Table  2  gives  the  complete  matrix  of  intercorrelations  among  all 
of  the  variables.  The  treatment  variable  has  significant  correlations 
with  Active  Time  on  Tasks  17  and  18  and  with  the  acceptance  measure.  The 
correlations  with  Active  Time  on  Task  17  and  with  acceptance  are  in  the 
predicted  direction  and  are  indicative  of  statistically  significant  treat¬ 
ment  effects.  Further  discussions  of  them  are  left  for  later  sections.  The 
remaining  independent  variables— education,  self-estimate  of  reading  level, 
and  ASVAB  GT  and  EL  are  all  significantly  correlated  with  each  other  and  they 
all  have  significant  correlations  with  Active  Time  on  Task  17  and  with  the 
knowledge  retention  test.  These  results  are  as  expected  and  attest  to  the 
correlational  validity  of  both  independent  and  dependent  measures. 

An  internal  consistency  estimate  of  reliability  (alpha)  of  .67 
was  obtained  for  the  knowledge  retention  test  using  a  combined  random 
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Means  and  Standard  Deviations  and  Sample  Sizes  of 
Independent  and  Dependent  Variables  For  Both 
Videodisc  and  Caramate  Groups 


Varlabi  e 

Videodisc 

Caramate 

N 

Mean 

SO 

N 

Mean 

Education 

72 

11 .79 

1.53 

162 

12.02 

Reading  Estimate 

72 

2.43 

.67 

162 

2.48 

ASVAB-GT 

55 

107.38 

12.54 

130 

107.91 

ASVAB-EL 

55 

109.47 

10.74 

130 

110.98 

Active  Time  Task  17 

72 

42.01 

15.55 

163 

46.31 

Active  Time  Task  18 

70 

27.97 

8.99 

158 

25.20 

Active  Time  Task  19 

69 

14.39 

10.16 

157 

13.40 

Fall  Task  17 

72 

.20 

.40 

163 

.19 

Fall  Task  18 

72 

.03 

.17 

163 

.06 

Fall  Task  19 

72 

.04 

.20 

163 

.04 

Knowledge  Retention 

72 

44.46 

5.61 

163 

44.81 

Acceptance  Questions 

72 

15.21 

1 .99 

163 

13.61 
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Alpha  less  than  or  equal  to  .05 


sample  of  50  slide-tape  students  and  25  videodisc  students.  Six  of  the 
40  items  had  multiple  responses  so  that  the  highest  possible  total  score 
was  59.  An  alpha  reliability  coefficient  of  .64  was  obtained  for  the  17 
common  items  of  the  acceptance  questionnaire.  The  total  sample  (n=235) 
was  used  for  this  computation. 

Evaluation  Question  #1 .  Will  students  who  learn  to  troubleshoot 
medium- powered  FM  radios  from  the  videodisc  delivery  system  attain  a 
statistically  significantly  lower  mean  progression  index  (^I)  than  that 
attained  by  students  who  learn  from  the  current  system? 

The  PI  was  computed  by  dividing  the  Active  Time  on  Task  17  for  each 
soldier  by  26.18,  the  assigned  time  for  the  task.  It  was  regressed  on  the 
dummy  coded  treatment  variable  (T),  the  education  variable  (E),  and  the 
interaction  of  treatment  and  education  (TE).  An  F-ratio  was  computed  on 
the  difference  between  the  squared  multiple  correlation  of  this  model  (.043) 
and  one  based  on  only  T  and  E  (.035)  and  was  found  not  to  be  significant  at 
the  .05  level.  The  interaction  term  was  not  significant  and  the  second 
model  was  retained.  The  results  for  it  are  shown  in  Table  3.  The  regression 
coefficients  for  both  T  and  E  were  significant  at  less  than  the  .05  level  and 
the  coefficient  to  T  shows  the  adjusted  difference  in  mean  Pi's  for  the  two 
groups.  The  first  evaluation  question,  then,  can  be  answered  affirmatively. 


Table  3 


Regression  Coefficients,  Standard  Errors,  F-Ratios, 
Levels  of  Statistical  Significance  and 
Multiple  Correlation  For  The  PI  of  Task  17  (Ng234) 


Variabl  e 

B 

S.E.B 

F 

Sig. 

Treatment 

-.178 

.081 

4.77 

.03 

Education 

-.060 

.029 

4.38 

.04 

Constant 

2.489 

.347 

51 .46 

<.01 

Multiple  R 

.19 

Evaluation  Question  #2.  Will  the  difference  in  mean  Pi's  of  the  two 
groups  be  of  practical  significance,  i.e.,  will  a  treatment  effect  size  of 
at  least  -.2  Pi's  be  attained?  The  PI  value  of  -.2  was  selected  as  being 
of  practical  significance  by  project  personnel  and  the  evaluator.  It 
represents  an  average  saving  in  active  student  time  of  slightly  more  than 
five  hours  given  the  assigned  time,  26.18  hours.  While  the  actual  value 
obtained,  -.178,  does  not  strictly  meet  the  criterion  for  practical 
significance  (the  savings  in  time  associated  with  it  is  4.7  hours), 
it  is  sufficiently  close  to  answer  the  question  affirmatively. 


However,  a  complicating  factor  in  the  interpretation  of  the  practical 
significance  of  the  treatment  effect  lies  in  the  results  of  the  analysis  of 
the  Active  Time  on  Task  18  as  shown  in  Table  4.  While  no  formal  evaluation 
question  was  concerned  with  it  or  with  Task  19,  analyses  of  them  were  planned 
to  determine  whether  transfer  effects  of  the  treatment  were  present.  Table  5 
shows  that  for  Task  19  no  significant  effects  were  found.  For  Task  18,  a 
regression  analysis  using  treatment  and  education  as  independent  variables 
found  only  a  significant  T  effect.  However,  the  regression  coefficient  for  T 

shows  that  the  actual  active  time  on  Task  18  for  videodisc  students  was  2.77 
hours  longer  than  for  Caramate  students.  There  is  no  obvious  reason  for  this 
result.  Plausible  interpretations  are  (1)  that  videodisc  students  suffered  a 
loss  of  motivation  upon  returning  to  the  Caramate  presentation  for  Task  18, 
(2)  that  the  use  of  instructional  (the  videodisc  player)  and  laboratory  (the 
radio)  equipment  in  the  same  physical  location  allowed  videodisc  trainees  to 
depend  on  the  videodisc  equipment  for  help  in  situations  where  Caramate 
trainees  would  have  had  to  rely  on  memory  or  their  own  notes,  thus  reducing 
to  some  extent  consolidation  or  overlearning  by  videodisc  students  of  the 
knowledge,  concepts,  and  skills  presented  in  Task  17.  The  second  interpreta¬ 
tion  would  imply  that  videodisc  trainees  had,  in  a  sense,  to  "catch  up"  with 
the  Caramate  group.  Additional  research  is  needed  to  determine  the  actual 
causes  of  this  finding  and  to  explore  its  implications  in  situations  where 
efficiency  in  terms  of  reduced  training  time  is  desired. 

Evaluation  Question  #3.  Will  videodisc  students  have  a  statistically 
higher  mean  on  a  knowledge  retention  test  than  that  of  Caramate  students?  A 
regression  analysis  of  the  knowledge  retention  test  using  treatment,  educa¬ 
tion,  and  their  interaction  as  independent  variables  failed  to  show  signifi¬ 
cant  treatment  or  T  by  E  effects.  These  results  are  also  shown  in  Table  6. 
This  question,  therefore,  is  answered  negatively. 

Evaluation  Question  #4.  Will  there  be  statistically  significantly 
fewer  students  in  the  videodisc  system  who  are  required  to  repeat  the  nwdule 
than  in  the  current  system?  Approximately  equal  proportions  of  students  in 
the  two  groups  failed  the  enabling  examinations  for  Task  17.  A  chi-square 
value  of  approximately  zero  was  calculated  from  these  data.  In  addition, 
chi-square  values  for  differences  in  failure  rates  in  the  two  groups  for 
Tasks  18  and  19  were  also  near  zero  and  nonsignificant.  Therefore,  this 
question  is  answered  negatively. 

Evaluation  Question  #5.  When  asked  to  indicate  the  degree  of  accept¬ 
ance  of  the  system  used  for  the  first  seven  lessons  of  the  medium- powered  FM 
radio  series,  will  students  who  receive  training  with  the  videodisc  system 
express  statistically  significantly  greater  acceptance  than  will  students  who 
receive  training  with  the  current  system?  A  regression  analysis  of  the 
acceptance  measure  with  treatment,  education  and  their  interaction  as  inde¬ 
pendent  variables  revealed  only  a  rignificant  positive  treatment  effect. 

The  results  are  shown  in  Table  7.  This  question  is,  therefore,  answered 
affirmatively.  As  an  aid  in  interpreting  this  question,  the  responses  of 
both  gtoups  were  tabulated  and  percentages  of  "yes"  responses  to  them  were 
calculated.  These  percentages  and  the  sample  sizes  on  which  they  were 
based  are  shown  in  Table  8.  It  should  be  remembered  that  the  acceptance 


Regression  Coefficients,  Standard  Errors,  F-Ratios 
Levels  of  Statistical  Significance  and 
Multiple  Correlations  For  The  Active  Time  on  Task  18  (N=225) 


Variabl  e 

B 

S.E.B 

F 

Sig. 

Treatment 

2.77 

1 .10 

6.33 

.01 

Education 

-.13 

.38 

.11 

.74 

Constant 

26.62 

4.66 

32.57 

<.01 

Multiple  R 

.17 

Table  5 

Regression  Coefficients,  Standard  Errors,  F-Ratios 
Levels  of  Statistical  Significance  and 
Multiple  Correlations  For  The  Active  Time  on  Task  19  (N=225) 


Variabl  e 

B 

S.E.B 

F 

Sig. 

Treatment 

.98 

1 .06 

.85 

.36 

Education 

.31 

.37 

.69 

.41 

Constant 

9.78 

4.49 

4.74 

.03 

Multiple  R 

.08 

Regression  Coefficients,  Standard  Errors,  F-Ratios, 

Levels  of  Statistical  Significance  and 
Multiple  Correlations  For  The  Knowledge  Retention  Test  (N=234) 


Variabl  e 

B 

S.E.B 

F 

Sig. 

Treatment 

-.20 

.72 

.08 

.78 

Education 

.53 

.25 

4.41 

.04 

Constant 

38.39 

3.07 

156.18 

<  .01 

Multiple  R 

.14 

Table  7 

Regression  Coefficients,  Standard  Errors,  F-Ratios, 
Levels  of  Statistical  Significance  and 
Multiple  Correlations  For  The  Acceptance  Measure  (N*234l 


Variabl  e 

B 

S.E.B 

F 

Sig. 

Treatment 

1 .58 

.31 

26.90 

<  .01 

Education 

-.07 

.11 

.44 

.51 

Constant 

14.46 

1 .30 

123.64 

<  .01 

Multiple  R 

.30 

Sample  Sizes  and  Percentages  of  Yes  Answers  Given  To 
Acceptance  Measure  Items  for  Both  Videodisc  and  Caranate  Groups 


Videodisc 


Caramate 


No.  Re> 
spending 

% 

Yes 

No.  Re¬ 
sponding 

% 

Yes 

1.  Were  you  comfortable  working  with  the 
videodisc  in  the  Lab? 

67 

96 

163 

83 

2.  Was  lighting  adequate  in  the  Lab? 

67 

100 

166 

95 

3.  Were  you  distracted  by  noise  in  the  Lab? 

66 

21 

164 

15 

4.  Did  you  have  enough  space  to  work  easily 
in  the  Lab? 

66 

62 

165 

67 

5.  Was  the  distance  from  the  videodisc 
player  about  right? 

67 

96 

155 

93 

6.  Was  it  too  close? 

42 

3 

96 

13 

7.  Was  it  too  far  away? 

40 

2 

94 

3 

8.  Were  the  written  materials  you  used  easy 
to  understand? 

64 

96 

159 

82 

9.  Did  they  use  too  many  unfamiliar  words? 

62 

3 

139 

12 

10.  Were  they  confusingly  organized? 

62 

12 

139 

26 

11.  Were  the  diagrams  easy  to  use  and 
understand? 

67 

96  1 

166 

93 

12.  Did  the  written  materials  and  diagrams 
go  together  well  for  easy  understanding? 

67 

1 

94 

161 

87 

13.  Were  all  printed  materials  large  enough 
and  clear  enough  for  easy  use? 

67 

93 

163 

95 

14.  Were  they  too  small? 

59 

9 

125 

8 

15.  Were  they  blurry  or  unclear? 

56 

2 

125 

16.  Was  the  speed  of  the  videodisc  about  right? 

66 

88 

156 

65 

17.  Did  it  go  too  fast? 

55 

2 

121 

18.  Did  it  go  too  slowly? 

55 

16 

125 

46 

19.  Were  you  bored  by  the  lessons? 

20.  Could  you  understand  all  that  the  speaker 

66 

12 

156 

said? 


67 


87 


159 


Table  8  continued 


Videodisc 

Caramate 

No.  Re- 

% 

No.  Re- 

% 

spending 

Yes 

spending 

Yes 

21  . 

Was  his  voice  dear? 

60 

93 

151 

52 

22. 

Did  he  use  words  that  you  did  not  know? 

61 

3 

139 

18 

23. 

Did  what  the  speaker  said  fit  with 
what  was  being  shown  on  the  screen? 

66 

100 

158 

87 

24. 

Did  materials  on  the  videodisc  go 
well  with  the  written  materials  you 

have  for  study  and  note  taking? 

67 

93 

165 

87 

25. 

Did  you  have  to  ask  the  instructor 
many  questions? 

66 

23 

162 

32 

26. 

Did  you  use  reference  notes  when  you 
worked  on  the  actual  radio  sets  in 

the  Lab? 

67 

97 

165 

96 

27. 

Did  you  make  new  notes  as  you  worked? 

64 

69 

150 

69 

28. 

Do  you  feel  confident  about  your 
performance  on  the  lessons  you  have 
compl eted? 

66 

97 

162 

91 

measure  was  based  on  only  the  seventeen  major  questions  in  the  table.  The 
questions  in  Table  8  are  those  received  by  the  videodisc  students.  The 
parallel  questions  asked  of  the  Caramate  students  are  not  given. 

Evaluation  Question  #6.  Will  students  who  experience  both  the 
videodisc  and  the  current  delivery  system  express  greater  satisfaction 
with  the  videodisc  system?  Videodisc  students  responded  to  fifteen  questions 
comparing  lessons  that  they  took  on  the  Caramate  system  with  the  ones  they 
took  on  videodisc.  The  results  are  shown  in  Table  9,  which  states  each 
question,  the  number  of  students  who  responded  to  it,  the  percentage  of 
students  who  gave  a  yes  answer  to  it,  and  the  chi-square  value  associated 
with  a  50:50  hypothesis  concerning  it.  It  can  be  seen  from  the  table  that 
students  were  generally  very  positive  about  the  videodisc  system  in  compari¬ 
son  with  the  Caramate. 

Evaluation  Question  #7.  Will  the  instructors  who  supervise  students 
undergoing  instruction  in  either  videodisc  or  current  systems  express  greater 
satisfaction  and/or  acceptance  of  the  videodisc  system  than  the  current 
system?  The  three  instructors  who  completed  the  questionnaire  reported 
without  exception  that  trainees  preferred  the  videodisc  to  the  Caramate 
delivery  system.  One  commented  that  the  videodisc  system  "holds  the  students' 
attention  more  completely"  and  that  students  were  "disappointed"  that  the 
"entire  course  was  not  on  videodisc." 

Affirmative  responses  were  obtained  to  questions  related  to  adequacy 
of  space  and  lighting  for  both  systems,  although  one  instructor  commented 
that  lighting  was  not  entirely  adequate  for  working  on  equipment. 

Two  of  the  three  instructors  reported  Caramate  students  as  requesting 
help  more  frequently  than  videodisc  students,  and  two  indicated  that  Caramate 
students  made  more  notes  for  later  use  than  did  videodisc  students. 

Only  one  instructor  said  that  one  group  seemed  better  able  to 
"zero  in"  on  specific  problem  areas  than  the  other. 

Answers  to  questions  related  to  student  confidence  were  split,  with 
only  one  instructor  reporting  videodisc  students  to  appear  more  "confident 
of  their  performance  levels,"  while  two  of  the  three  said  that  videodisc 
students  had  "greater  confidence  in  handling  the  radio  equipment  in  the  Lab." 

No  statistical  analysis  could  be  made  for  such  a  small  number  of 
responses.  However,  a  conclusion  that  instructor  acceptance  of  the  video¬ 
disc  system  is  generally  favorable  seems  to  be  warranted. 

Reliability  and  maintainability  of  the  videodisc  system.  During  the 
entire  trial  period  only  one  equipment  failure  (a  laser  slide)  occurred 
after  an  elapsed  time  of  160.3  hours.  This  mechanical  failure  resulted  in  a 
critical  mission  stoppage  for  that  machine.  Nine  days  occurred  between  the 
failure  and  repair  of  the  machine,  with  labor  costs  of  $117.60,  parts  costs 
of  $257.16,  and  administrative  costs  of  $7.35.  No  scheduled  maintenance 
was  performed  on  any  machine  during  the  approximately  ten  weeks  of  operation. 


Table  9 


Responses  of  Videodisc  Students  to  Questions 
Comparing  Videodisc  and  Caramate  Lessons 


No.  Re¬ 
sponding 

% 

Yes 

Chi- 

Square 

1 . 

Did  you  like  using  the  videodisc  better 
than  you  liked  using  the  Caramate? 

67 

90 

41 .93* 

2. 

Did  you  think  you  learned  more  quickly 
with  the  videodisc 

67 

89 

35.84* 

3. 

Were  diagrams  as  sharp  and  clear  on  the 
videodisc  as  on  the  Caramate? 

65 

94 

24.99* 

4. 

Did  the  physical  movement  (motion)  on  the 
videodisc  player  make  course  materials 
learned  easier  to  understand  than 
materials  learned  with  the  Caramate 
player,  which  has  only  still  pictures? 

66 

96 

54.55* 

5. 

Did  you  use  the  backward  motion  capabil ity 
of  the  videodisc  to  review  or  clarify 
course  material ? 

67 

91 

45.15* 

6. 

Did  you  find  it  helpful? 

62 

94 

47.03* 

7. 

Did  you  use  the  slow  moving  speed  capa¬ 
bility? 

67 

34 

6.58* 

8. 

Did  you  find  it  helpful? 

54 

43 

.30 

9. 

Did  you  use  the  rapid  search  of  course 
material  capability? 

66 

96 

54.55* 

10. 

Did  you  find  it  helpful? 

64 

98 

60.06* 

11 . 

Did  you  use  the  videodisc  when  you  were 
working  on  actual  equipment? 

67 

72 

12.55* 

12. 

Did  you  also  use  reference  notes? 

66 

94 

50.97* 

13. 

Did  you  make  new  notes  at  that  time? 

66 

68 

8.73* 

14. 

Did  you  ask  your  instructor  more  questions 
when  you  used  the  videodisc  than  when  you 
used  the  Caramate? 

65 

6 

49.98* 

15. 

Do  you  feel  more  confident  about  your  per¬ 
formance  on  materials  learned  with  the 
videodisc  than  on  those  learned  with  the 
Caramate? 

65 

89 

29.50* 

*  Alpha  equal  to  or  less  than  .05 


Summary/Conci usions 


Training  materials  for  Task  17,  the  first  seven  lessons  of  module 
eight  of  the  fourteen  module  Tactical  Communications  System  Operator/ 
Mechanic  Course  were  administered  to  two  groups  of  trainees  using  two 
different  delivery  systems.  Specific  materials  used  were  concerned  with 
troubleshooting  medium- powered  FM  radios  of  the  AN/VRC-12  series.  One  group 
of  trainees  used  the  current  delivery  system,  which  is  a  Caramate  slide 
projector;  the  other  group  received  instruction  with  a  videodisc  system. 

A  total  of  235  trainees  were  randomly  assigned  to  groups,  72  to  use  video¬ 
disc  equipment,  and  163  to  use  the  Caramate  system.  Relative  training 
effectiveness/efficiency  of  the  two  systems  was  studied  through  comparison 
of  mean  progression  indices,  mean  scores  on  a  knowledge  test,  and  proportion 
of  trainees  required  to  repeat  the  module  for  the  two  groups.  Comparative 
acceptability  of  the  systems  was  determined  through  administration  of 
questionnaires  to  students  in  both  groups  and  to  instructors  for  the  course. 

Records  of  maintenance  performed/ equipment  breakdowns  repaired  were 
kept  for  use  in  determining  relative  reliabilities  and  cost  of  maintenance 
for  equipment  for  the  two  systems. 

Trainees  in  the  Caramate  group  used  the  delivery  system  normally  in 
the  individual  learning  center  (ILC)  and  laboratories.  Differences  in 
training  conditions  for  the  videodisc  group  were  that,  1)  they  received 
introductory  instruction  for  using  the  equipment,  2)  they  used  the  different 
equipment,  3)  they  worked  in  the  Lab  with  both  videodisc  equipment  and  the 
AN/VRC-12  radio,  and  4)  they  used  materials  that  had  been  rewritten  to  take 
advantage  of  special  capabilities  of  the  equipment.  Content  of  the  lessons 
and  enabling  and  criterion  examinations  were  the  same  for  both  groups. 

Regression  analysis  was  the  major  method  used  in  analyzing  data 
related  to  training  effectiveness.  Data  from  student  questionnaires  were 
also  analyzed  by  this  method.  Instructor  acceptance  data  were  not  subjected 
to  statistical  analysis  because  of  the  small  number  (3)  of  instructors 
involved. 

Trainees  who  used  the  videodisc  delivery  system  had  a  significantly 
lower  mean  PI  on  Task  17  than  did  trainees  using  the  Caramate  system;  the 
net  gain  for  videodisc  trainees  over  Caramate  trainees  for  Task  17  v<as 
4.7  hours.  However,  on  Task  18  videodisc  students  had  a  significantly 
greater  mean  completion  time  U.8  hours).  No  significant  differences 
between  groups  were  found  for  either  proportion  of  trainees  required  to 
repeat  the  module  or  mean  scores  on  a  knowledge  test. 

Both  trainees  and  instructors  showed  high  degrees  of  acceptance  of 
the  videodisc  system,  preferring  it  to  the  Caramate  system.  Only  one 
equipment  failure  was  reported,  and  no  scheduled  maintenance  was  performed 
during  the  comparison  period.  Thus,  no  results  were  obtained  to  indicate 
comparative  costs/ probl ems  with  maintaining  the  two  systems. 

While  these  results  are  somewhat  equivocal,  the  overall  conclusion 
is  that  the  videodisc  system  was  superior  to  the  Caramate  system  in  training 


efficiency  and  acceptance,  but  that  no  evidence  of  its  superiority  in  training 
effectiveness  could  be  found.  That  is,  trainees  in  the  two  groups  achieved 
similar  levels  of  knowledge  of  the  course  content  and  experienced  similar 
success/failure  rates.  However,  those  in  the  videodisc  group  took  signifi¬ 
cantly  less  time  to  complete  the  task  and  that  time  reduction  closely  approxi¬ 
mated  the  level  set  to  indicate  practical  importance.  The  fact  that 
videodisc  trainees  took  longer  than  Caramate  trainees  to  complete  Task  18  is 
of  little  importance  if  it  can  be  interpreted  to  mean  that  students  are  less 
efficient  when  they  are  changed  from  a  preferred  system  to  a  less  preferred 
one.  It  is  expected  that  similar  results  would  be  found  if  the  study  were 
to  be  repeated  under  the  same  conditions.  Because  of  the  complex  nature  of 
the  study,  the  exact  reasons  for  the  superiority  of  the  videodisc  system 
cannot  be  determined.  Therefore,  generalizations  about  its  effectiveness  in 
other  situations  cannot  be  made.  If,  for  example,  the  study  were  repeated 
with  the  ILC  and  the  Lab  combined  for  Caramate  students  as  it  was  for  video¬ 
disc  students,  the  difference  in  mean  Pi's  might  disappear.  Also,  if  the 
videodisc  system  were  to  be  implemented  for  the  whole  course,  its  novelty 
effect  would  be  reduced  and  the  mean  PI  might  be  unaffected. 

On  the  other  hand,  the  superiority  of  the  videodisc  system  could 
well  be  due  to  its  greater  flexibility  in  presenting  course  material.  Over 
ninety  percent  of  the  videodisc  students  reported  using  the  backward  motion 
and  rapid  search  features  of  the  videodisc  and  finding  them  to  be  helpful 
features.  It  seems  unlikely  that  these  positive  reports  were  due  to  the 
novelty  of  the  system  since  only  thirty- four  percent  of  the  trainees  reported 
using  the  slow  speed  capability  and  only  forty-three  percent  of  those  reported 
it  to  be  hel pful . 

Recommendations 

While  this  study  was  too  limited  to  warrant  a  recommendation  that 
the  videodisc  system  be  implemented  for  the  entire  course,  it  is  valuable 
in  that  its  results  can  be  combined  with  those  of  other  studies  conducted 
under  different  conditions  of  setting,  content  materials,  and  trainee 
samples  to  arrive  at  more  dependable  conclusions.  It  is  recommended  that 
meta-analysis  be  used  to  evaluate  the  combined  results  of  many  studies  such 
as  this  one  to  produce  a  broader  and  much  more  general izable  picture  of  the 
effectiveness  of  videodisc  delivery  of  instruction.  Special  attention  should 
be  given  to  determining  whether  other  studies  contain  evidence  that  training 
methods  that  increase  efficiency  by  reducing  training  time  also  show  evidence 
of  possible  reduction  of  knowledge/skills  consolidation.  If  few  or  no 
studies  have  investigated  this  phenomenon,  specific  research  designed  to 
study  it  in  training  and  situations  involving  job  performance  should  be 
conducted.  Finally,  it  is  recommended  that  comparisons  be  made  of  the  re¬ 
sults  of  videodisc-alone  studies  with  those  of  studies  using  other  educa¬ 
tional  technologies,  specifically  studies  of  microcomputer- videodisc 
delivery  systems,  in  order  to  determine  the  most  cost-effective  system  for 
instruction  of  Army  trainees. 
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Appendix  A 


DIRECTION  TO  STUDENT:  Answer  all  of  the  questions  on  this  test.  Do  not 
refer  to  your  Guidance  Package  or  to  any  Technical  Manual  while  taking  the 
test.  When  finished,  return  your  copy  of  the  test  to  the  Test  Monitor. 


Write  your  name  and  SSN  below,  along  with  the  date. 


(4-4-5)  T  F  With  the  R-442  in  its  NEW  ON  squelch  mode,  the  CALL  lamp 
will  light  when  the  receiver  is  turned  off  by  the  squelch 
circuit . 

(4-4-1 )  T  F  The  R-442  receives  an  FM  signal  in  the  same  way  as  the 
receiver  portion  of  the  associated  RT  unit. 

(7-1-2)  T  F  One  of  the  differences  between  matching  units  MX-6707  and 
MX-2799  is  that  the  MX-6707  has  a  frequency  selector  knob. 

(1-7-1)  Which  auxiliary  component  is  used  for  relay  operation  when  the 
radio  set  is  an  AN/VRC-45  or  -49? 

a.  AN/GRA-39 

b.  C-2742 

c.  C-2299 

d.  MX-6707 

(3-2-6)  Responsibil ity  of  N.C.S.  is: 

a.  tactical  control 

b.  technical  control 

c.  direct  control 

d.  indirect  control 

(4-1-1)  Put  these  preparatory  steps  in  the  proper  order.  Place  a  "1" 
by  the  step  which  is  first  and  a  "2"  by  the  second,  and  so 
forth.  Write  your  answers  in  this  book. 

Set  the  MX  6707  to  the  RT  dial  setting 

_ Cable  the  RT  to  include  the  AN/URM-182 

_  Preset  the  RT's  controls 

_  Verify  the  radio  set's  installation 

_  Preset  the  R-442  controls 

_  Turn  all  switches  off  to  avoid  injury 

(1-7-3)  Which  auxiliary  component  is  used  to  remote  the  radio  set  up  to 
two  miles  using  wire? 

a.  AN/GRA-39 

b.  C-2742 

c.  C-2299 

d.  MX-6707 


•* •  •'*»  .  •’»  •  »  •  k  ■  •  •** 


(4-2-3)  Match  the  terms  in  Column  A  with  the  description  in  Column  B, 


A.  Terms  3.  Descriptions 

Squelch  sensitivity  a.  Lowering  the  audio  level  of  a 
Call  light  receiver  when  a  transmitter  is 

Keying  being  keyed. 

Muting  b.  Refers  to  the  amount  of  incoming 

signals  required  to  automatically 
turn  the  receiver  sound  on. 

c.  A  receive  signal  indicator. 

d.  Pressing  the  push-to-talk  switch. 

The  R-442  has  two  squelch  modes:  NEW  ON — used  when  all 
radios  of  the  net  are  capable  of  transmitting  a  150  Hz 
tone  and  OLD  ON — used  when  one  or  more  of  the  radios 
of  the  net  cannot  transmit  a  150  Hz  tone. 


(4-4-3) 


(2-4-8)  The  best  thing  you  can  use  to  clean  the  audio  plugs  and 
jack  is  a: 


a.  wet  finger 

b.  silicon  gel 

c.  pencil  eraser 

d.  cleaning  cloth 


(1-6-2)  What  are  the  numbers  for  the  following  configurations  of  the 
AN/VRC-12  series? 


12.  (1-5-9)  Match  the  matching  units  in  column  B  with  the  appropriate 

antenna  system  in  column  A. 

Antenna  system  Matching  unit 

_ AT- 91 2  a.  MX-6707 

_ AS-1729  b.  MX-2799 

_  R-442  antenna  c .  AB-1 5 

13.  (2-7-4)  Which  auxiliary  component  is  used  to  extend  the  audio 

accessories  within  the  vehicle  in  which  the  radio  set 
is  mounted? 

a.  AN/6RA-39 

b.  C-2742 

c.  C-2299 

d.  MX-6707 

14.  (7-1-3)  T  F  The  MX-6707  and  MX-2799  are  two  types  of  antenna 

matching  units  used  with  AN/VRC-12  series  radios. 


Refer  to  the  pin  voltage  chart  above  and  answer  the  following  true 
and  false  questions. 

15.  (7-3-10)  T  F  For  proper  operation  of  the  matching  units,  pin  N  of 

J2  must  have  22  to  30  VDC  applied. 


17.  (4-2-2) 


18.  (7-1-1) 


(3-4-4) 


the  description  In  Column  B. 

B.  Descriptions 

Listening  to  a  radio  signal . 
Automatically  quieting  a 
receiver  when  no  radio 
signal  condition  exists. 

Sound  used  to  Initially 
check  an  FM  receiver. 

The  circuit  that  delivers 
battery  voltage  to  the 
radio  component. 

That  part  of  the  RT's 
antenna  system  that  changes 
the  antenna  tuning. 

The  purpose  of  the  matching  units,  MX-6707  and  MX-2799,  Is  to 

a.  Tune  the  RT  to  a  proper  dial  frequency. 

b.  Tune  the  antenna  according  to  the  RT's  dial  frequency. 

c.  Tune  the  receiver's  antenna,  AB-15. 

d.  Tune  the  receiver  to  a  proper  dial  frequency. 

What  is  the  phonetic  alphabet  equivalent  for  the  letter  "C"? 

a.  Cheerful 

b.  Charlie 


Match  the  terms  in  Column  A  with 
A.  Terms 

_  Power  input  a. 

_  Rushing  noise  b. 

_  Matching  unit  control 

_ Squelch 

_  Reception  c. 

d. 


e. 


DIRECTION:  Refer  to  the  diagram  of  the  AN/VRC-47  above  to  answer  the 
following  questions. 

20.  (7-2-1)  Place  an  X  by  each  component  or  cable  that  Is  part  of  the 

matching  unit's  DC  control  circuit. 


MX-2799  CX-4722 
MT-1029  MX- 6707 
R-442  CX-4721 
RT-524  (RT-246)  _ C6-1773 


21.  (5-1-6)  T  F  Cable  CX-4720  Is  connected  to  the  mount  MT-1029  at 

jack  J-23. 

22.  (5-2-7)  T  F  Cable  CG-1773  Is  connected  between  the  RT-524  antenna 

coaxial  jack  and  the  antenna  matching  network  coaxial 
jack. 

23.  (6-1-5)  T  F  Within  the  MT-1029,  the  negative  side  of  the  source  Is 

connected  to  pin  A  of  jacks  J21 ,  J22,  J23,  and  J24. 

24.  (5-3-7)  T  F  The  RT-524  transmitted  RF  signal  passes  through  cable 

CX-4722  to  the  antenna  AS-1729. 


25.  (5-3-1)  T  F  The  microphone  M-80  is  normally  connected  to  the  audio 

jack  labeled  SPKR. 

26.  (6-1-8)  T  F  To  energize  the  RT's  relay  K404,  a  ground  path  must  be 

completed  from  pin  2  of  K404,  through  the  contacts  of 
the  power  switch. 

27.  (5-3-10)  T  F  Cable  CX-4722  is  used  to  pass  DC  source  voltage  from 

the  RT-524  to  the  AS-1729. 

28.  (5-3-5)  T  F  The  transmit  signal  patch  for  the  RT-524  begins  at 

the  M-80. 

29.  (5-3-3)  T  F  When  the  M-80  is  keyed,  the  transmitter  section  relays 

should  click  and  the  blower  motor  should  run. 

30.  (5-2-3)  T  F  From  the  antenna  jacks  on  the  R-442  the  signal  passes 

through  the  receiver  R-442  to  the  audio  jacks  on  its 
front  panel . 

31.  (6-1-3)  T  F  For  the  mount  MT-1029,  the  negative  side  of  the  starts 

at  jack  J21 ,  pin  A. 

32.  (5-3-9)  T  F  DC  source  voltage  is  applied  to  different  pins  of  the 

antenna  control  connector  of  the  RT-524  as  the  frequency 
of  the  RT-524  is  changed. 

33.  (6-3-1)  T  F  Source  voltage  to  the  receiver,  R-422,  starts  at  the 

mount,  MT-1029,  jack,  J22. 

34.  (5-1-3)  T  F  Source  voltage  through  cable  CX-4721  is  applied  to 

the  MT-1029. 

35.  (5-1-2)  T  F  Source  voltage  from  the  MT-1029  is  applied  to  the  RT-524 

36.  (6-3-4)  T  F  Source  voltage  is  applied  to  pin  C  of  P201  through  the 

receiver  R-422  to  light  the  dial  lamp. 

37.  (5-2-2)  T  F  Cable  CG-1773  is  connected  to  the  coaxial  connector 

of  the  mast  base  AB-15  to  either  antenna  coaxial  jacks 
on  the  R-442. 

38.  (6-1-6)  T  F  The  MT-1029's  jack  024  mates  with  the  RT's  plug,  P401 , 

to  apply  source  voltage  to  the  RT  (pins  B  and  0). 

39.  (5-2-9)  T  F  The  audio  accessory  H-140  is  normally  connected  to  the 

mike  jack  labeled  SPKR  on  the  front  panel  of  the 
RT-524. 
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(2-4-3)  The  RF  cable  CG-1773  is  connected  between  the  antenna  jack 
on  the  AB-15  and: 

a.  the  RF  antenna  jack  of  Rt-524 

b.  the  antenna  control  jack  of  RT-524 

c.  the  left  RF  antenna  jack  of  R-442 

d.  the  right  RF  antenna  jack  of  R-442 

e.  either  one  of  the  RF  antenna  jacks  of  R-442 


For  Caramate  Delivery  System  Students  (CDSS) 


Please  answer  the  following  questions  about  the  lessons  you  have  completed 
with  AN/VRC-12  Series  Radio  Sets: 

1.  Were  you  comfortable  working  in  the  Lab?  Yes _  No 

Comments: _ _ _ 


2.  Was  lighting  adequate  in  the  Lab? 
Comments :  _ 


3.  Were  you  distracted  by  noise  in  the  Lab? 
Comments : _ _ 


Yes  No 


4.  Did  you  have  enough  space  to  work  easily  in  the  Lab?  Yes _ No. 

Comments : _ 


5.  Was  the  distance  from  the  Caramate  screen 


about  right? 

Yes _ 

_  No. 

Was  it  too  close? 

Yes _ 

_  No. 

Was  it  too  far  away? 

Yes _ 

_ No 

Comments 


12.  Could  you  understand  all  that  the  speaker  said? 


Yes 


No 


J*'C’** 


Was  his  voice  clear? 

Old  he  use  words  that  you  did  not  know? 
Comments : 


13.  Did  what  the  speaker  said  fit  with  what  was 
being  shown  on  the  screen? 

Comments : 


Did  materials  on  the  Caramate  go  well  with 
the  written  materials  you  have  for  study 
and  note  taking? 

Comments : 


15.  Did  you  have  to  ask  the  Instructor  many  questions? 
Comments : 


Did  you  use  reference  notes  when  you  worked  In  the 


Did  you  make  new  ones? 
Comments : 


Yes  No 


igssggc^y 


17.  Do  you  feel  confident  about  your  performance 


on  the  lessons  you  have  completed? 
Comments ; _ _ _ 


Please  answer  the  following  questions  about  your  work  In  the  ILC: 

18.  Were  you  comfortable?  Yes _ No 

Comments : _ _ _ _ 


19.  Was  1  Ighting  adequate?  Yes _ No 

Comments ; _ _ 


20.  Were  you  distracted  by  noise?  Yes _ No _ 

Comments ;  _ 


21 .  Did  you  have  enough  space  to  work  easily?  Yes _ No. 

Comments : _ _ _ 


For  Videodisc  Delivery  System  Students  (VDSS) 


Please  answer  the  following  questions  about  the  lessons  yoii  have  completed 
with  AN/VRC-12  Series  Radio  Sets: 

1.  Were  you  comfortable  working  with  the  videodisc 

in  the  Lab?  Yes _ No 

Comments : 


2.  Was  lighting  adequate  in  the  Lab? 
Comments : 


Yes _ No 


3.  Were  you  distracted  by  noise  in  the  Lab?  Yes _  No 

Comments :  _  _ 


4.  Did  you  have  enough  space  to  work  easily  in  the  Lab?  Yes _  No 

Comments : 


Was  the  distance  from  the  videodisc  player  about  right? 

Yes _ 

No 

Was  it  too  close? 

Yes _ 

_  No _ 

Was  it  too  far  away? 

Yes_ 

No 

Comments ; 
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1.  Were  you  bored  by  the  lessons? 
Comments : _ 


2.  Could  you  understand  all  that  the  speaker  said? 
Was  his  voice  clear? 

Did  he  use  words  that  you  did  not  know? 

Comments :  _ 


13.  Did  what  the  speaker  said  fit  with  what  was  being 
shown  on  the  screen? 

Comments ; _ _ _ 


14.  Did  materials  on  the  videodisc  go  well  with  the 

written  materials  you  have  for  study  and  note  taking? 
Comments :  _ _ _ 


15.  Did  you  have  to  ask  the  instructor  many  questions? 
Comments : _  . 


16.  Did  you  use  reference  notes  when  you  worked  on  the 


actual  radio  sets  in  the  Lab? 

Did  you  make  new  notes  as  you  worked? 


17.  Do  you  feel  confident  about  your  performance  on 


the  lessons  you  have  coii4)leted?  Yes _ Ho 

Comments : _ _ _ 


Please  answer  the  following  questions  comparing  lessons  you  studied  with  the 
Caramate  with  those  you  studied  with  the  videodisc: 

\ 

18.  Did  you  like  using  the  videodisc  better  than  you 

liked  using  the  Caramate?  Yes _ No _ 

Comments :  _ _ _ _ _ 


19.  Did  you  think  you  learned  more  quickly  with 

the  videodisc?  Yes _ No 

Comments : _ 


20.  Were  diagrams  as  sharp  and  clear  on  the  videodisc 

as  on  the  Caramate?  Yes _ No 

Comments : _ _ 


21.  Did  the  physical  movement  (motion)  on  the  videodisc 
player  make  course  materials  learned  easier 
to  understand  than  materials  learned  with  the 

Caramate  player,  which  has  only  still  pictures?  Yes _ No 

Comments :  _ 


26.  Did  you  ask  your  instructor  more  questions  when 

you  used  the  videodisc  than  when  you  used  the  Caramate? 
Comments :  _ 


27.  Do  you  feel  more  confident  about  your  performance 
on  materials  learned  with  the  videodisc  than  on 
those  learned  with  the  Caramate? 


Comments : 


Lab  Instructor  Questionnaire 


Please  answer  the  following  questions  about  study  conditions  in  the  Lab. 

1.  Do  student  stations  in  the  Lab  provide 
adequate  space  for  working  with  both  the 

videodisc  and  radio  equipment?  Yes _  No 

2.  Is  lighting  adequate  for  working  with  the 
videodisc  as  well  as  for  working  with  radio 

equipment?  Yes _ No 

For  the  following  questions,  please  compare  the  Lab  performance  of 
students  who  have  worked  with  the  Caramate  delivery  system  in  the 
ILC  with  those  who  have  worked  in  the  Lab  using  the  videodisc  system: 

3.  Have  students  in  one  group  requested  help  from 
the  instructor  more  frequently  than  those  in 

the  other?  Yes _ No 

If  yes,  which  group?  Caramate _  Videodisc 

4.  Have  students  in  one  group  seemed  to  zero  in  better 

on  specific  areas  where  they  need  help?  Yes _  No 

5.  Which  system  did  students  seem  to  like  better?  Caramate _  Videodisc 

6.  Did  it  appear  that  one  group  made  more  reference 

notes  than  the  other  for  later  use  in  the  field?  Yes _  No 

If  yes,  which  group?  Caramate _ Videodisc 


7.  Did  one  group  of  students  seem  to  be  more 
confident  of  their  performance  levels  than 


i 

the  other? 

•'j 

If  yes,  which 

i 

8.  Did  one  group 

o. 

confidence  in 

V. 

in  the  Lab? 

If  yes,  which 

Comments : 

L 


group? 

appear  to  have  greater 
handling  the  radio  equipment 

group? 


Yes _ Mo 

Caranate  Videodisc 


Yes _ No 

Caramate _ Videodisc 
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tha  rapair  of  this  tgulpaant. 


Naintananca  Parts  Cost  •  Entar  tha  cost  of  rapair  parts  mqulrad  to  rapair  this 

odulpant. 


Adaln  Hours 


AMa  Coats 


Catagory  of  mintana 


•  Entar  tha  nunbar  of  hours  raquirad  to  adalnlstratlvaly 
prooasa  tha  ogulpaont. 


•  EMmp  tha  east  of  adarfnl strati valy  prtMassIng  this 
saulpaant. 


•  Fdr  ttrfs  data  tha  aalntananca  tachnielan  will  e* 
of  tha  fallowing  eatagorlas: 

Schadulad  -  If  this  was  schadulad  saint' 
this  block  * 

UaschadBlad  •  If  this  was  unsehadulaf 
dMcfc  this  block 


Typo  of  Mlntanancs 


Parts/Nodula 


chock  tha  following  typos  of  ns' 


EA  •  Claetrlea)  Adjust 
EX  •  Elactrleal  Rsplaes 
QA  •  Optical  Adjust 
OK  •  O^cal  Roplacs 
NA  •  NMianleal  Adjust 
NR  •  Nadtanleal  Raplaca 

bitar  tha  naas  of  tha  part/nodul 
raquirad  rapair. 


BELIMILITY  AND  MAIWTAINABtLITY  DATA  QEFTWITIOWS  FOR  EIDS  EVALUATIONS 

Thtrt  is  no  strict  format  for  filling  out  this  data  stioot.  Just  insurt  that 
avorything  is  Itgiblo  and  oonmct.  If  you  dosiro  to  usa  a  dacimal  point  within  any 
nuMr,  accantuata  it  so  that  it  is  claarly  visibla. 

n«f  of  Palluro  •  Entar  tha  data  that  tha  fallura  was  first  obsarvad. 

Exaapla:  25  Oac  79. 

Maka/Modol  -  Plaasa  just  antar  tha  naka/modal  of  tha  systam  baing 

avaluatad. 

Srlal  huabar  •  Entar  tha  Sarial  NuadMr  of  tha  playar  that  has-fallad  or 

raquiras  pravantativa  aalntananca. 


Wfcw/Madal 


ETH  Hours 
Criticality 


Entar  tha  hours  ragistaring  on  tha  alapsa  tina  natar  at 
tha  tima  of  tha  fallura  or  pravantativa  aalntananca. 

Oiaek  “yas*  If  this  fallura  is  a  critical  Mission  stoooaoa 
fallura.  Chack  “no"  If  this  fallura  is  not  a  c^ti cal 
mission  fallura.  Tha  dafinitlon  of  tha  nrisslon  fallura  is 
ona  that  would  nafca  training  on  this  systn  laeossibla. 
Criticality  will  ragulra  a  subjactlva  dacislon  by  tha  ust 
monitor.  As  an  axampla.  fbr  nost  lassons  tha  ust  of  color 
Is  not  ragulrad  and  tha  loss  of  color  would  not  stop  tha 
systM  from  dallvarlng  training  natarlals;  houavar,  eartain 
training  asttrlals  raipilra  color  and  tha  loss  of  that  color 
would  rasult  In  a  critical  fallura  of  tha  systas  to  dallvar 
Its  IntHidad  mfsslon  •  training.  Tha  monitor  will  naka  his 
dKislon  on  a  casa  by  casa  basis. 


•  For  this  data  tha  nonltor  will  chack  all  catagorias  that 
•gplys 

PfM  >  Pravantativa  Naintahanca  Adjust 
PMR  •  Pravantativa  Maintananea  naplaca 
PMC  •  Pravantativa  Maintananea  Claan 
Schadulad  -  If  this  was -schadulad  oparator  pravantativa 
aalntananca  chack  this  block 

.  Unachadulad  •  If  this  was  unschadulad  oparator  pravant* 
ativa  maintananea  chack  this  block 

•  Plaua  chack  tha  following  typas  of  maintananea  that  apply: 

EA  •  ETactrical  Adjust 
ER  •  ETactrical  Raplaea 
M  •  Optical  Adjust 
OR  •  Optical  Raplaea 
m  -  Noehanlcal  Adjust 
NR  -  Noehanlcal  Raplaea 
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Appendix  0 


This  appendix  gives  selected  computer  output  of  the  regression 
analyses.  On  the  first  page,  the  SPSS^  procedure  file  is  listed.  Line  5 
gives  the  list  of  variables: 

T  Treatment  (1  “videodisc,  0*Caraniate) 

E  Self  report  by  trainee  of  highest  level  of  education 

he/she  attained. 

R  Estimate  of  reading  by  trainee  on  a  5-point  scale. 

G  GT-ASVAB  score. 

EL  EL-ASVAB  score. 

A7-A9  Active  time  in  hours  on  Tasks  17,  18,  19. 

P  Overall  course  PI  (not  used  in  analyses). 

PF7-PF9  Number  of  failures  on  enabling  examinations  for  Tasks  17, 
18,  19. 

K  Knowledge  Retention  Test  score. 

A  Acceptance  Questionnaire  score. 

The  remaining  lines  provide  the  following  Information  and  instructions: 

Line  10  -  Input  format  for  the  data  file. 

Line  15  •  Instruction  to  consider  blanks  as  missing  data. 

Line  20  •  Set  all  pass/fall  values  to  1  or  0. 

Line  25  -  Compute  PI  for  Task  17. 

Line  30  -  Compute  treatment  by  education  interaction  variable  (TE) 

Line  35  -  Instruction  to  consider  blanks  in  PI  and  TE  as  missing 
data. 

Line  40  -  Regress  PI  on  T,  E,  and  TE.  Regress  PI  on  T  and  E. 

The  output  listing  at  the  bottom  of  the  first  page  shows  the  results 
of  regressing  PI  for  Task  17  on  T,  E,  and  TE.  The  overall  F-ratio  was 
statistically  significant  but  no  one  of  the  main  effects  or  the  interaction 
was  significant.  This  resulted  because  of  the  high  degree  of  colinearity 
of  the  interaction  with  the  main  effect  terms. 

The  output  listed  at  the  top  of  the  second  page  shows  the  results 
of  regressing  PI  for  Task  17  on  only  the  main  effects  T  and  E.  Both 
effects  are  statistically  significant  and  the  difference  of  the  adjusted  PI 
means  is  -.178. 


^Nie,  N.  H.,  Hull,  C.  H.,  Jenkins,  J.  G.,  Steinbrenner,  K.,  and 
Bent,  D.  H.  Statistical  Package  for  the  Social  Sciences  (2nd  ed.;  New 
York:  McGraw-Hl1K  l970).  ^ 


The  remaining  listings  are  for  the  analyses  of  active  time  on 
Tasks  18  and  19,  the  Knowledge  Retention  test  and  the  Acceptance  Question¬ 
naire.  Only  the  results  of  the  analyses  involving  the  two  main  effects  are 
given.  Analyses  for  these  dependent  variables  that  included  the  interaction 
term  all  produced  findings  that  were  essentially  the  same  as  those  for  the 
PI  for  Task  17. 


SPSS/ONLINE  AUTO-HODE 
list 

5.  VARIABLE  LIST 

5.005  T.EiRt0>ELiA7>A8fA9>P.PF7>PF8fPF9>K»A 
10.  INPUT  FORHAT 

10.005  FIXED<4XrF1.0tF3.0iF2.0>2F4.0>3F5.0>F4.0»lX>3F1.0r2F3.0) 
15.  MISSING  VALUES 
15.005  T  TO  A(BLANK> 

20.  RECODE 

20.005  PF7  TO  PF9(0>0) (BLANK-BLANK) (ELSE- 1) 

25.  COMPUTE 
28.005  PI-A7/2A18 
30.  COMPUTE 
30.005  TE  -TtE 
38.  ASSIGN  MISSING 
35.005  PI>TE(BLANK) 

40.  REGRESSION 

40.005  VARIASLE8«TfEtR>Pt»TE/ 

40.00A  REORESSION-Pl  WITH  T>E»TE/RE0RE38I0N>PI  WITH  T.E/ 

40.007  REGRESSION-PI  WITH  TtE»R 


OEP.  VAR...  PI 

MEAN  RESPONSE  1.71790  STD.  DEV.  .57987 


FINAL  STEP. 


MULTIPLE  R 

.2083 

ANOVA 

OF  SUN 

SOUARES 

MEAN  SO. 

F 

R  SQUARE 

.0434 

REGRESSION 

3. 

3.397 

1.132 

3.47 

STD  DEV 

.5709 

RESIDUAL 

230. 

74.950 

.326 

SIG.  .01 

ADJ  R  SQUARE 

.0309 

COEFF  OF  VARIABILITY 

33.2PCT 

VARIABLE 

B 

3.E.  B 

F 

SIG. 

BETA  ELASTICITY 

T 

.778 

.494 

1.288 

.263 

.62058 

.13935 

E 

-.027 

.037 

.507 

.477 

-.06021 

-.18484 

TE 

-.080 

.058 

1.924 

.167 

-.76801 

-.16976 

CONSTANT 

2.088 

.451 

21.422 

.000 

aIl  VARIABLES  ARE  IN  THE  EQUATION. 


3EP.  VAR...  PI 

mean  response 

PIMAt  StEP. 

HOLTIPtiB  R 
R  sau^E 
STO  DEO 


1.71790 


.IQSO  ANOtl»A 
.0333  RESRESSION 
.^720  RESIDUAt 


STD.  DEV. 


.57987 


OF  SUN  SQUARES  HiAff  SQ. 

2.  2.770  1.385 

l3i.  75.577  .327 


AOJ  R  StiUARE  .0270  eOSFP  OF  VARiMtiLiTY  33i3PET 


N  SQ.  P 

1.385  4.233 

.327  sis.  .016 


VARIAtLE 


CONSTANT 


S.Ei  • 


-.178 

.081 

4.772 

-.060 

.029 

4.387 

2.489 

.347 

51.463 

AcL  VARIABLES  ARE  IN  THE  EQUATION. 


BETA  elasticity 


-.14164 

-.13580 


'.03181 

'.41689 


HEP.  VAR...  A8 

MEAN  RESPONSE  2593.79111 

FINAL  STEP. 


STO.  8EV. 


768.83736 


miLTIPLE  R  .1701  ANOVA  DF  SUN  SOUAREB  MEAN  SO.  F 

R  SOUARE  .0289  REORESStSN  2.  3829Ae8»095  .19t607  3.30< 

STO  OEV  761.0425  RESIOSAL  222.  .iStBi^dO  .59C4>06  SIO.  .031 

AOJ  R  SQUARE  .0202  COEFF  OF  VARIABILITY  QP.SPCT 


VARIABLE 


B 

S.E.  B 

•  F 

die. 

277.611 

110.375 

6.326 

.013 

-12.854 

38.433 

.112 

.738 

2662.503 

466.547 

32.568 

.000 

BETA  ELASTICITY 

.16687  .03282 

-.02219  -.05931 


ULl  variables  are  in  the  equation. 


■.V-is 


4451 


OEP .  ^AR . . . 


A9 


MEAN  RESPONSE  1374.10222  STD.  DEW.  731.94939 

FINAL  STEP. 

MULTIPLE  R  .0799  ANOWA  OF  SUN  SQUARES  MEAN  SO.  F 


R  SQUARE 

.0044 

REGRESSION 

2.  745348.247 

.38E-<-04 

.71; 

STD  OEV 

732.9109 

RESIOUAL 

1192E+09 

.S3E-t-04 

SIO.  .49; 

AOJ  R  SQUARE  0 

COEFF  OF  VARIABILITY 

53.3PCT 

VARIABLE 

B 

S.E.  B 

F 

SXQ. 

BETA  ELASTICITY 

T 

97.893 

104.295 

.848 

.358 

.04181 

.02182 

E 

30.703 

37.002 

.489 

.408 

.05549 

.24705 

CONSTANT 

97S.S94 

449.301 

4.744 

.030 

^  WARXASLCS  ARE  IN  THE  EQUATION* 


OEP.  VAR...  K 

MEAN  RESPONSE  44.41374  STO.  OEV.  S.092S3 

FINAL  STEP. 

MULTIPLE  R  .1400  ANOVA  OF  SUN  SQUARES  MEAN  SO.  F 

R  SQUARE  .0194  RCSRESSION  2.  llS.4f3  59.247  2.310 

STO  OEV  5.0441  RESXOUAL  231.  5924*105  25.445  SIB.  .102 

AOJ  R  SQUARE  .0111  COEFF  OF  VARIAIILXTY  11.3PCT 

VARIABLE  B  S.E.  B  F  SIO.  BETA  ELASTICITY 

T  -.202  .720  .071  .790  -.01S31  -.00139 

E  .532  .253  4.414  .037  .13734  .14235 

CONSTANT  3S.3SS  3.071  154*113  0 


ALW  VARIABLES  ARE  IN  THE  EQUATION 


OEP.  IMR...  A 


MEAN  RESPONSE 

14.10254 

STD.  DEV. 

2.24139 

FINAL  STEP. 

MULTIPLE  R 

.3293 

ANOVA 

OF  SUM 

SQUARES 

MEAN  SQ. 

F 

R  SQUARE 

.1084 

REBRESSION 

2. 

129.170 

64.585 

14.043 

STD  DEV 

2.144S 

RESIDUAL 

231. 

1042.349 

4.599 

SIG.  .000 

ADJ  R  SQUARE 

.  1007 

COEFF  OF  VARIABILITY 

15.2PCT 

VARIABLE 

B 

S.E.  B 

F 

SIO. 

BETA  ELASTICITY 

T 

l.SSl 

.305 

24.900 

0 

.32331 

.03449 

E 

CONSTANT 

-.071 

14.443 

.107 

1.301 

.434 

123.443 

oo 

• 

-.04117 

-.04004 

ALL  UARIAtLES  ARE  XN  THE  EQUATtON. 


.‘.’.S  .N 
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